Wavefront detector

ABSTRACT

A wavefront sensor system suitable for integration into an integrated circuit light detector may provide for wave angle sensors having varying functional relationships between the wave angle and signal to provide improved dynamic range. These wave angle sensors may be combined with integrated circuit phase angle sensors for a more complete analysis of the waveform.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

This invention was made with government support under DE-NA0002915awarded by the U.S. Department of Energy. The government has certainrights in the invention.

CROSS REFERENCE TO RELATED APPLICATION

NA

BACKGROUND OF THE INVENTION

The present invention relates generally to imaging devices and inparticular to wavefront angle or wavefront phase sensitive imagingdevices.

Conventional digital cameras record light intensity from an illuminatedscene, for example, using electronic sensors distributed over a focalplane, to each sense a “pixel” of the image to produce the image. Byincorporating colored filters over the electronic sensors, lightfrequency as well as intensity may be recorded providing colored images.

Conventional digital cameras do not capture important information aboutthe light received from an illuminated scene that may be expressed inthe light's wavefront angle or phase. “Light field” or “plenoptic”cameras, however, can capture this wavefront information together withlight intensity information providing a more complete representation ofthe illuminated scene. As a result, such cameras can change the focaldistance and depth of field after the image is captured. More generally,the captured wavefront information provides a more complete record ofthe illuminated scene that may be useful in a variety of applicationsincluding postprocessing, image recognition, hologram generation, andthe like.

The image sensors using conventional digital cameras, such as chargecoupled devices (CCD), do not naturally detect wavefront information butcan be modified to produce a Shack-Hartman type wavefront detector. Suchmodifications place an array of micro lenses in front of a standardimage sensor, for example, each micro lens being associated with atwo-dimensional zone of multiple pixels of the image sensor. Lightreceived by each micro lens provides a focal spot whose position on themultiple pixels of the zone changes depending on the angle of thewavefront. By detecting which pixels detect the greatest light intensity(indicating the location of the focal spot) the wavefront angle may bededuced. The measured intensity of this focal spot is also used toprovide the conventional image intensity information.

SUMMARY OF THE INVENTION

The present invention provides a wavefront detector employing “shadowcasters” instead of micro lenses greatly simplifying the construction ofa Shack-Hartman type wavefront detector, for example, directly on anintegrated circuit substrate. Further, by providing shadow casters ofdifferent heights, the sensitivity of the wavefront detectors can bevaried to effect an improved trade-off between wavefront anglesensitivity and range of wavefront angle detection.

In some embodiments, the shadow casters may be combined with integratedcircuit light mixers generating constructive and destructiveinterference between light received at adjacent pixels to deduceabsolute phase difference. This absolute phase difference can be usedalone or combined with wave angle measurements for improved wavefrontanalysis.

A measurement of wavefront angle (for example, using a Shack-Hartmantype wavefront detector) and a measurement of absolute phase difference(for example, using light mixing structures) will collectively bereferred to as “wavefront detection” or as using a “wavefront sensor”herein.

Specifically, then, the present invention provides a wavefront sensorhaving an array of light intensity sensor elements tiling a substrate toreceive light, the light having a wavefront angle with respect to asurface normal of the substrate. A set of shadow casters is providedwhere each shadow caster is associated with a group of at least twosensor elements to selectively shade different sensor elements of thatgroup as a function of the wavefront angle. The different shadow castershave different heights above the substrate with respect to the surfacenormal to provide different shadow lengths along the substrate as afunction of wavefront angle.

It is thus a feature of at least one embodiment of the invention toprovide improved sensitivity in wavefront angle measurement whileavoiding “clipping” or saturation that can cause the loss of informationat high wavefront angles.

The wavefront sensor may include a circuit comparing light intensitymeasured by different sensor elements in a group to provide an outputsignal related to wavefront angle, and the circuit may apply a first orsecond predetermined function to light intensity measurements of thedifferent sensors of the group depending on the height of the shadowcaster associated with the group to provide the output signal.

It is thus a feature of at least one embodiment of the invention toaccommodate arbitrarily complex changes in the functional relationshipbetween brightness and wavefront angle for different shadow casterheights.

Each group of light sensing elements may be associated with a colorfilter, and the circuit may apply a different predetermined function tolight intensity measurements of the different sensors of the groupdepending on the filter color.

It is thus a feature of at least one embodiment of the invention tocorrect for changes in the functional relationship between wavefrontangle and sensor reading that occurs with different frequencies as notedby the inventors.

The group of sensor elements may be adjacent and separated over twodimensions along the substrate.

It is thus a feature of at least one embodiment of the invention topermit two dimensions of wavefront angle determination.

The shadow caster may be a perimeter wall surrounding a group of sensingelements.

It is thus a feature of at least one embodiment of the invention toprovide a simple structure for fabrication using integrated circuittechniques.

The perimeter wall may partially cover the sensing elements.

It is thus a feature of at least one embodiment of the invention topermit ready adjustment of the sensitivity of the wavefront sensor bycontrolling the exposed area of the sensor elements where smallerexposed areas provide increased angular sensitivity.

The sensing elements may be displaced beneath a surface of thesubstrate.

It is thus a feature of at least one embodiment of the invention toprovide a system that can work with common image sensing architecturesin which the light sensors are positioned beneath a circuitry andconnection layer.

The shadow casters of different heights are uniformly distributed overthe surface of the substrate.

It is thus a feature of at least one embodiment of the invention toallow both high dynamic range and high sensitivity measurementsthroughout the image area.

The wavefront sensor may further include a set of mixers each associatedwith at least two sensor elements to receive light from adjacentseparated locations along the plane of the substrate and to mix thatlight together before providing mixed light independently to each of theat least two sensor elements. The intensity of the light provided to theat least two sensor elements varies in intensity as a function of phasedifference of light received by the mixer at the separated locations.

It is thus a feature of at least one embodiment of the invention toprovide a wavefront sensor that can reveal absolute phase differencesbetween sensing locations.

Each mixer may provide a first and second inlet light pipe leading fromrespective adjacent separated locations and communicating with a commoncavity and may further provide a first and second outlet light pipeleading from the cavity to two different sensor elements.

It is thus a feature of at least one embodiment of the invention toprovide a structure that can be simply integrated into an integratedcircuit image sensor.

The mixer may be a transparent semiconductor material of the substrateembedded with in a substrate material a different index of refraction.

It is thus a feature of at least one embodiment of the invention toprovide a wavefront sensor using readily available integrated circuitmaterials.

Different ones of the set of mixers maybe associated with pairs ofadjacent separated locations displaced along different dimensions of theplane of the substrate.

It is thus a feature of at least one embodiment of the invention toprovide phase difference measurements in two orthogonal directions toprovide a better reconstruction of two-dimensional wavefronts.

The circuit may further compare light intensity measured by the at leasttwo sensor elements associated with a mixer to provide an output equalto a phase difference between light received at the adjacent separatedlocations.

It is thus a feature of at least one embodiment of the invention toprovide a decoding of phase information to be added to other informationfrom the image sensor.

The circuit may combine the phase differences from the mixer with thewavefront angle to provide a description of a wavefront received at thesubstrate.

It is thus a feature of at least one embodiment of the invention toprovide a more complete description of the wavefront by combiningwavefront angle and absolute phase differences.

Some sensor elements of the substrate are associated with shadow castersand do not have mixers and some sensors of the substrate have mixers andare not associated with shadow casters.

It is thus a feature of at least one embodiment of the invention toprovide wavefront angle and wavefront phase measurements in the singlesensor system.

These particular objects and advantages may apply to only someembodiments falling within the claims and thus do not define the scopeof the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a fragmentary, exploded perspective view of a portion of animage sensor showing individual sensor elements in regular rows andcolumns positioned behind a shadow casting mask for identification ofwavefront angle;

FIG. 2 is a cross-section along line 2-2 of FIG. 1 showing the effect ofthe shadow generated by the shadow casting mask in changing exposedareas of the sensor elements as may be measured to deduce wavefrontangle;

FIG. 3 is a plot of a ratio of the intensity of the light in the pixelelements of FIG. 2 as a function of angle in a simplifiedtwo-dimensional example such as may be used by circuitry to create anangle measurement from this ratio information;

FIG. 4 is a figure similar to FIG. 2 showing a shadow caster having ahigher elevation from the sensor substrate such as produces greatersensitivity to wavefront angle;

FIG. 5 is a figure similar to that of FIG. 3 showing a plot of the ratioof the intensity of light in the pixel elements of FIG. 4 as a functionof angle as compared to the same plot of FIG. 3 showing the change insensitivity commensurate with the reduction in measurement range;

FIG. 6 is a fragmentary top plan view of an image sensor incorporatingthe shadow casters of FIGS. 2 and 4 to provide both high sensitivity andhigh range type sensors;

FIG. 7 is a cross-sectional view similar to FIGS. 3 and 4 showinginterposition of colored filters, for example, used for color imaging,to produce signals applied to different functions converting theintensity ratios into wavefront angles for the different colors;

FIG. 8a-c are fragmentary perspective views similar to that of FIG. 1but unexploded showing alternative shadow casters including a circularmask in FIG. 8a , a post structure in FIG. 8b , and a partial wallstructure in FIG. 8 c;

FIG. 9 is a figure similar to FIG. 1 showing an alternative wavefrontsensor design providing absolute phase difference measurements betweenadjacent pixels using an integrated circuit-mounted light mixingelement;

FIG. 10 is a figure similar to FIG. 6 showing interconnection of themixers at various image points to provide two-dimensional phasedifference measurements;

FIGS. 11a and 11b are example waveforms received by the sensor systemsof FIGS. 1 and 9 respectively showing interpretation of that data; and

FIG. 12 is a top plan view of an integrated circuit substrate for lightsensing similar to FIGS. 6 and 10 showing a hybrid sensor elementincorporating both wavefront angle and wavefront phase sensing.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring now to FIG. 1, a wavefront detector 10 of the presentinvention may provide a wavefront angle sensor 11 working in conjunctionwith a standard integrated circuit image sensor 12 having an array oflight sensing elements (pixels 14) arranged in rows and columnsseparated by gutters 16 on an integrated circuit substrate 18.Generally, the light sensing elements may be photodiodes eitherpositioned adjacent to an upper planar surface of the substrate 18 or ona lower layer of the substrate 18 (as depicted) with light passingthrough upper layers of the substrate 18 to reach the photodiodes. Inthis latter case, processing circuitry and intercommunicating conductors20 may be placed between the pixels 14, for example, in the gutters 16.In either case, the pixels 14 are sensitive to light passing along (notnecessarily parallel to) an axis 22 aligned with a surface normal of thesubstrate 18 and provide independent electrical signals indicating lightreceived within the area of the pixel 14.

Referring also to FIG. 2, a shadow-casting mask 24 may be placed overthe pixels 14 to block some but not all incoming light along axis 22.Generally, the mask 24 includes an opening 26 exposing portions of fouradjacent pixels 14 positioned around a shared centerpoint 15 in thecenter of the mask 24. The mask 24 exposes a detection zone 28 of theadjacent pixels 14.

In this embodiment, the mask 24 provides a square frame that coversportions of each of the pixels 14 along an outer periphery of thatdetection zone 28. The mask 24, for example, may be a metallizationlayer and may have many different openings 26 defining correspondingmultiple zones 28 over the entire area of the image sensor 12 havingmany pixels 14.

Referring now to FIG. 3, light passing along a light axis 25 at an angleα with respect to axis 22 and through centerpoint 15 will generate ashadow 30 over at least one pixel 14 a caused by an inner wall of theopening 26, the latter extending along axis 22. The particular pixel 14a being shadowed will be pixel 14 whose surface forms an acute anglewith respect to the light axis 25.

The shadow 30 on the pixel 14 a and the mask 44 decreases a total areaof illumination of this pixel 14 a to a reduced area 32.

Conversely pixel 14 b who surface forms an obtuse angle with respect tolight axis 25 does not have a shadow 30 and provides a larger exposurearea 34 equal to the full area of the exposed portion of the pixel 14 bthrough the mask 24 as well as a leakage area 36 of light passingunderneath the mask 24 into the pixel 14 b. In addition, some portion oflight entering the surface of pixel 14 a near axis 22 with incidentangle α can travel long enough to reach the pixel 14 b.

Electrical signals 38 from the pixels 14 a and 14 b may be received bycomparison circuitry 40 to deduce the wavefront angle 43 (angle α) byapplying a ratio formed from the light intensity measured by pixels 14 aand 14 b to an empirically described transform curve 42. While in thissimple example only a single dimension of pixels 14 a and 14 b is shown,this angle can be readily extended to two perpendicular directions byconsidering the ratio of the intensity of light in each of the fourpixels 14 of the zone 28. So, for example, a ratio between the twocolumns of pixels 14 may determine light angle in one direction, and aratio between the two rows of pixels 14 may determine a light angle inthe perpendicular direction.

Referring now to FIG. 3, this transform curve will generally show aratio moving between a value of one for angles of α equal 0 and tohigher or lower angles as a ratio deviates above and below 1. Thistransformation between a ratio value and an angle α may be accomplishedthrough discrete circuitry or through a microprocessor executing asimple program accessing a lookup table or performing the calculationbased on an established formula. These same electrical signals 38 may bereceived by conventional image processing circuitry 45 for thegeneration of an intensity image 46 as is understood in the art.

Referring now to FIG. 4, some zones 28 may employ a thicker mask 44having a larger height measured along axis 22 than the height of mask 24shown in FIG. 2. This taller mask 44 will provide larger shadows 30 fora given angle α; however, it will generally not change the leakage areas36. Nevertheless, the proportionally greater shadow 30 will produce agreater sensitivity to values of a as shown by transform curve 50compared to transform curve 42. On the other hand, this greatersensitivity means that the transform curve 50 can operate in a narrowerrange of angle α since the angle α where the shadow 30 will cover thewhole pixel 14 a is smaller when mask 44 is used instead of mask 24.

Referring to FIG. 6, the present invention contemplates that at leasttwo thicknesses of masks 24 and 44 may be used on different zones 28 ofwavefront detector 10 to provide a trade-off between high sensitivityand high dynamic range. For example, masks 24 and 44 may be alternatedalong given rows and columns of the zones 28. Each of these zones 28will be associated with circuitry using either transform curve 42 or 50as is appropriate for the height of the mask 24 or 44 associated withthat zone 28. The invention further contemplates that masks of multipledifferent thicknesses (not just two) may be used if desired, again, withcorresponding transform curves.

Referring now to FIG. 7, a color camera may be implemented byassociating colored filters 54 a-54 c (typically red, green, and blue)with each zone 28 of pixels 14. The inventors have determined that suchfiltration can modify curves 42 and 50 and, accordingly, a set ofdifferent curves (three associated with masks 24 and three associatedwith masks 44, one mask for each color) may be used by the circuitry 40in deducing angle α. A combined signal from the pixels 14 of each zone28, according to their particular filter 54, may also be sent to theimage processor 45 for the development of a standard color image. Thecircuitry 40 completes the wavefront angle sensor 11.

Referring now to FIG. 8a , it will be appreciated that the mask 24 or 44may have a circular opening rather than the square opening of opening 26shown in FIG. 1 to provide improved symmetry but at the cost of reducedexposed pixel area. In some embodiments, the mask 24 and 44 may beplaced totally over the gutter area to reduce blockage of the pixels 14.As shown in FIG. 8b , other shadow casting structures may be readilyused including a post 60 or other gnomon structure extending upwardalong axis 22, for example, centered within the zone 28 of four pixelswith a shadow 30 falling on one or more of the pixels 14 depending onthe angle of the light axis 25 of the incoming light. As shown in FIG.8c , the post may be compressed to a cruciform lying completely withinthe gutter 16 of the light sensor. As before, the height of thestructures may be adjusted to effect a trade-off between sensitivity anddynamic range as discussed above.

Referring now to FIG. 9, a phase difference sensor 64 may be constructedindicating an absolute phase difference between the adjacent locationsof light reception on the wavefront detector 10. The phase differencesensor 64 provides phase information rather than waveform angle providedby the wavefront angle sensor 11 discussed above. The difference betweenthese types of sensors will be discussed in more detail below withrespect to FIG. 11.

In this embodiment, a mask structure 66 (for example, a metallizationlayer) may be positioned over the pixels 14 providing a sample aperture68, for example, centered over each pixel 14 and otherwise blockinglight transmission to the pixel 14. Light from the adjacent apertures 68(which will be displaced laterally from each other) may be received bycorresponding light pipes 70 of a mixer 72 formed in the integratedcircuit material of the substrate 18 beneath the mask structure 66. Forexample, the light pipes 70 may be constructed from transparent siliconedioxide surrounded by a material of higher index of refraction toprovide a reflective interface.

The light pipes 70 may join to a light cavity 74 of similar material sothat the light from each of the light pipes 70 mixes together toconstructively and destructively interfere. After this mixing, the lightis then conducted out of exit light pipes 76 (of similar material to thelight cavity 74) carrying this light to respective pixels 14 a and 14 b,for example, being adjacent along a given axis of the array and beneaththe corresponding aperture 68.

The cavity 74 may be designed as an interferometer in the manner of thecombining portion of a Mach-Zehnder interferometer. More generally, thelight exiting from the different light pipes 76 will have differentintensities based on different interference paths in the light cavity 74from the light received by the two entrance light pipes 70. For example,a rightmost light pipe 76 may sum together some light frequenciesreceived by the light pipe 70 whereas the leftmost light pipe 76 maysubtract those frequencies. More generally, the output from the lightpipes 76 will be different complex summations of the light from the twoentrance light pipes 70 with varying frequency offsets.

The intensity of light measured by the pixels 14 a and 14 b may becompared (for example, using a ratio) and this comparison applied to anempirically derived transform curve 80 to produce a phase differencesignal 82 indicating the phase difference between the lights arriving atthe aperture 68. This light may be pre-filtered by colored filters asdiscussed above. More generally a phase difference signal 82 may begenerated by a multidimensional function directly receiving intensitysignals 38 from the pixels 14.

Referring momentarily to FIG. 10, the sample apertures 68 may extendregularly in two dimensions over the surface of the substrate 18 and maybe pairwise connected by mixers 72 along two perpendicular directions ofthe substrate 18, for example, in a serpentine path as depicted, toprovide phase difference measurements into orthogonal directions. Inthis regard each aperture 68 may feed two different mixers 72.

Referring now to FIGS. 11a and 11b , the difference between measurementof a wavefront angle 43 (per the wavefront angle sensor 11 shown in FIG.2) and a phase difference measurement (per the phase difference sensor64 shown in FIG. 9) can be understood by reviewing an example waveform84 having a step phase shift 86. A set of spaced measurement zones 28that can detect only wavefront angle α, for example, per the sensorsshown in FIGS. 1-8 and will provide identical measurements of wavefrontangle 43 at each zone 28 based on the incident of wavefront angles atthe zones 28 resulting in a reconstructed waveform 88 blind to the stepphase shift 86, as shown in FIG. 11 a.

In contrast, as shown in FIG. 11b , a sensor of the type shown in FIG. 9measuring a change in phase angle will accurately reflect the step phaseshift 86 in a reconstructed waveform 88 made from measurements at zones28, albeit without capturing the wavefront angle 43. By combining thesetwo types of data, an improved waveform can be developed approximatingwaveform 84 by enforcing the necessary angles on the wavefront at theangle measurement points and the necessary phase differences at thephase difference points.

Referring now to FIG. 12, the benefit of both of these different typesof measurements of wavefront angle and absolute phase difference may beprovided by alternating or otherwise distributing wavefront anglesensors 11 and phase difference sensors 64 over the surface of thesubstrate 18 to blend these types of measurements to create a moreaccurate rendition of the incoming waveform. It will be appreciated thatlight collected by the wavefront angle sensor 11 and phase differencesensors 64 may also be used in the construction of the standard image,for example, by tapping into the signals as shown in FIG. 2. Inaddition, the wavefront angle sensors 11 may include both sensor masks24 and 44 to provide improved dynamic range.

Certain terminology is used herein for purposes of reference only, andthus is not intended to be limiting. For example, terms such as “upper”,“lower”, “above”, and “below” refer to directions in the drawings towhich reference is made. Terms such as “front”, “back”, “rear”, “bottom”and “side”, describe the orientation of portions of the component withina consistent but arbitrary frame of reference which is made clear byreference to the text and the associated drawings describing thecomponent under discussion. Such terminology may include the wordsspecifically mentioned above, derivatives thereof, and words of similarimport. Similarly, the terms “first”, “second” and other such numericalterms referring to structures do not imply a sequence or order unlessclearly indicated by the context.

When introducing elements or features of the present disclosure and theexemplary embodiments, the articles “a”, “an”, “the” and “said” areintended to mean that there are one or more of such elements orfeatures. The terms “comprising”, “including” and “having” are intendedto be inclusive and mean that there may be additional elements orfeatures other than those specifically noted. It is further to beunderstood that the method steps, processes, and operations describedherein are not to be construed as necessarily requiring theirperformance in the particular order discussed or illustrated, unlessspecifically identified as an order of performance. It is also to beunderstood that additional or alternative steps may be employed.

References to “a microprocessor” and “a processor” or “themicroprocessor” and “the processor,” can be understood to include one ormore microprocessors that can communicate in a stand-alone and/or adistributed environment(s), and can thus be configured to communicatevia wired or wireless communications with other processors, where suchone or more processor can be configured to operate on one or moreprocessor-controlled devices that can be similar or different devices.Furthermore, references to memory, unless otherwise specified, caninclude one or more processor-readable and accessible memory elementsand/or components that can be internal to the processor-controlleddevice, external to the processor-controlled device, and can be accessedvia a wired or wireless network.

It is specifically intended that the present invention not be limited tothe embodiments and illustrations contained herein and the claims shouldbe understood to include modified forms of those embodiments includingportions of the embodiments and combinations of elements of differentembodiments as come within the scope of the following claims. All of thepublications described herein, including patents and non-patentpublications, are hereby incorporated herein by reference in theirentireties.

What we claim is:
 1. A wavefront sensor comprising: an array of lightintensity sensing elements tiling a substrate to receive light having awavefront angle with respect to a surface normal of the substrate; a setof shadow casters, each shadow caster associated with a group of atleast two sensing elements to selectively shade different sensingelements of that group as a function of the wavefront angle; whereindifferent shadow casters have different heights above the substrate withrespect to the surface normal to provide different shadow lengths alongthe substrate as a function of wavefront angle; and a circuit comparinglight intensity measured by different sensing elements in a group toprovide an output signal related to wavefront angle.
 2. The wavefrontsensor of claim 1 wherein the circuit applies at least a first andsecond predetermined function to light intensity measurements of thedifferent sensors of the group depending on the height of the shadowcaster associated with the group to provide the output signal.
 3. Thewavefront sensor of claim 2 wherein the group of light sensing elementsis associated with a color filter and wherein the circuit applies adifferent predetermined function to light intensity measurements of thedifferent sensors of the group depending on a color of the filter. 4.The wavefront sensor of claim 1 wherein the group of sensing elementsare adjacent and are separated over two dimensions along the substrate.5. The wavefront sensor of claim 1 wherein the shadow caster is aperimeter wall surrounding a group of sensing elements.
 6. The wavefrontsensor of claim 5 wherein the perimeter wall partially covers thesensing elements.
 7. The wavefront sensor of claim 1 wherein the sensingelements are displaced beneath a surface of the substrate.
 8. Thewavefront sensor of claim 1 wherein shadow casters of different heightsare uniformly distributed over the surface of the substrate.